Skip to content

HYPERFLEET-549 - feat: standard configuration#71

Open
rh-amarin wants to merge 2 commits intoopenshift-hyperfleet:mainfrom
rh-amarin:HYPERFLEET-549
Open

HYPERFLEET-549 - feat: standard configuration#71
rh-amarin wants to merge 2 commits intoopenshift-hyperfleet:mainfrom
rh-amarin:HYPERFLEET-549

Conversation

@rh-amarin
Copy link
Copy Markdown
Contributor

@rh-amarin rh-amarin commented Mar 11, 2026

Summary

  • Standardises the sentinel configuration schema by adding yaml:"..." struct tags to all config types, producing consistent snake_case field
    names both in config files and in config-dump output.
  • Adds a config-dump CLI subcommand that prints the fully-merged configuration (file + env vars + CLI flags) as YAML, making it easy to
    inspect the effective config at runtime or in CI.
  • Adds sentinel.name and the build version to the User-Agent header sent on every HyperFleet API request hyperfleet-sentinel/<version> (<name>).
  • Adds a comprehensive config-loading test suite (test/integration/test-config-loading.sh, 83 assertions) that verifies every parameter
    loads correctly from all three sources (file, env var, CLI flag) and that CLI > env > file precedence is respected.
  • Adds an integration test (TestIntegration_ConfigLoadingScript) that runs the shell test suite inside a bash:5 testcontainer against a
    freshly cross-compiled Linux binary, so the full config path is exercised on Linux in CI.
  • Updates Helm chart, example configs, and adds docs/configuration.md and CONFIGURATION_MIGRATION.md for operators migrating from the old
    flat schema.

Test plan

  • make build — binary compiles cleanly
  • go test ./internal/config/... ./internal/client/... ./internal/sentinel/... — all unit tests pass
  • ./test/integration/test-config-loading.sh --verbose — all 83 shell tests pass against the local binary
  • make test-integration -run TestIntegration_ConfigLoadingScript — shell tests pass inside a Linux container
  • sentinel config-dump -c configs/dev-example.yaml — prints merged config with correct snake_case field names and User-Agent visible in API
    request logs

Summary by CodeRabbit

  • New Features

    • Added a config-dump command and CLI override flags to view the merged runtime configuration; merged config can be printed with sensitive values redacted.
    • Client now includes sentinel/version metadata in requests (User-Agent).
  • Configuration

    • Restructured config to nested, snake_case keys; expanded clients.hyperfleet_api and broker settings (including broker.topic).
    • Environment variables renamed (BROKER_TOPIC → HYPERFLEET_BROKER_TOPIC; LOG_* → HYPERFLEET_LOG_*).
    • Added logging, debug-config, and API retry/backoff options; message_data key standardized.
  • Documentation

    • Added a Sentinel Configuration Reference and updated running/deployment docs and chart README.
  • Tests

    • Added extensive config-loading unit tests and an integration config-loading script.

@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Mar 11, 2026

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Mar 11, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign rh-amarin for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 11, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

This PR restructures configuration into a nested model under clients, adds sentinel metadata, logging and debug sections, and renames many keys from camelCase to snake_case. HyperFleet API settings move to clients.hyperfleet_api, broker topic nests under clients.broker.topic, and environment variables gain a HYPERFLEET_ prefix. LoadConfig now merges CLI flags, env vars, and files with explicit precedence and validation; a new config-dump CLI command prints the merged, redacted YAML. The HyperFleet client constructor now accepts sentinel name and version to set a User-Agent. Helm charts, examples, tests, and docs updated accordingly.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant CLI_Flags as "CLI Flags"
    participant Env as "Environment Vars"
    participant Config_File as "Config File"
    participant Loader as "config.LoadConfig"
    participant Logger as "initLogging"
    participant App as "Sentinel (serve / config-dump)"
    participant Hyperfleet as "HyperFleet Client"

    User->>CLI_Flags: provide flags (optional)
    User->>Env: set env vars (optional)
    User->>Config_File: provide config file (optional)

    CLI_Flags->>Loader: flag values
    Env->>Loader: env values
    Config_File->>Loader: file values

    Loader->>Loader: merge (flags > env > file > defaults)
    Loader->>Loader: validate & produce redacted copy

    Loader->>Logger: pass Log config
    Logger->>App: initialize logging

    Loader->>Hyperfleet: supply base_url, timeout, sentinel name, version
    App->>Hyperfleet: initialize client (User-Agent set)

    opt config-dump
      User->>App: run config-dump
      App->>Loader: load merged config
      App->>User: print redacted YAML
    end

    opt serve
      User->>App: run serve
      App->>App: start watchers & broker using merged config
    end
Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 61.73% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies this as a configuration standardization feature with the issue reference HYPERFLEET-549.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@rh-amarin rh-amarin force-pushed the HYPERFLEET-549 branch 3 times, most recently from 96cde3b to 37a82bc Compare March 12, 2026 17:28
@rh-amarin rh-amarin marked this pull request as ready for review March 12, 2026 17:42
@openshift-ci openshift-ci bot requested review from mbrudnoy and tirthct March 12, 2026 17:42
@rh-amarin
Copy link
Copy Markdown
Contributor Author

rh-amarin commented Mar 12, 2026

@coderabbitai review

@rh-amarin
Copy link
Copy Markdown
Contributor Author

@coderabbitai help

@rh-amarin rh-amarin marked this pull request as draft March 12, 2026 19:32
@rh-amarin rh-amarin marked this pull request as ready for review March 12, 2026 19:32
@openshift-ci openshift-ci bot requested review from 86254860 and aredenba-rh March 12, 2026 19:32
@rh-amarin
Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@rh-amarin rh-amarin force-pushed the HYPERFLEET-549 branch 2 times, most recently from d30dabd to a4e1d5f Compare March 12, 2026 22:37
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

♻️ Duplicate comments (1)
cmd/sentinel/main.go (1)

379-379: ⚠️ Potential issue | 🟠 Major

config-dump currently emits merged config in plaintext.

Marshaling cfg directly can leak sensitive values (for example from header maps) into terminal/CI logs. Defaulting to a redacted copy is safer.

Suggested safer default
-	data, err := yaml.Marshal(cfg)
+	data, err := yaml.Marshal(cfg.RedactedCopy())
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@cmd/sentinel/main.go` at line 379, The config-dump currently marshals cfg
directly which can leak secrets; instead create and marshal a redacted copy
(e.g., call a helper like redactConfig(cfg) or cfg.Redacted()) that copies the
config and replaces sensitive fields (authorization/header maps, API keys,
tokens, secret strings, TLS private keys, etc.) with placeholder values before
calling yaml.Marshal; update the code around the data, err := yaml.Marshal(cfg)
line to use the redacted copy so terminal/CI logs never contain raw secrets.
🧹 Nitpick comments (1)
charts/values.yaml (1)

109-113: Unused field may confuse users.

config.clients.broker.topic is defined here but never used in charts/templates/configmap.yaml. The template at line 40 sources the topic from .Values.broker.topic (line 159) instead.

Consider either:

  1. Removing this field entirely, or
  2. Using this field in the configmap template instead of .Values.broker.topic

This would prevent users from editing the wrong field when configuring their topic.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@charts/values.yaml` around lines 109 - 113, The values.yaml defines
config.clients.broker.topic but the configmap template
(charts/templates/configmap.yaml) is reading .Values.broker.topic, causing the
former to be unused and confusing users; either remove
config.clients.broker.topic from values.yaml or update the configmap template to
read the topic from .Values.config.clients.broker.topic (replace occurrences of
.Values.broker.topic with .Values.config.clients.broker.topic) so the intended
field is actually applied.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@configs/dev-example.yaml`:
- Line 26: Update the stale example comment that reads "Logging configuration
(can be overridden by LOG_* env vars or --log-* flags)" to reference the actual
env-var prefix "HYPERFLEET_LOG_*" (and, if relevant, the CLI flag prefix) so the
comment becomes something like "Logging configuration (can be overridden by
HYPERFLEET_LOG_* env vars or --log-* flags)"; locate the string in the file (the
exact comment text) and replace the LOG_* token with HYPERFLEET_LOG_*.

In `@configs/gcp-pubsub-example.yaml`:
- Line 21: Update the logging comment text so the env-var prefix matches the
current config contract: replace the referenced `LOG_*` token in the comment
string with `HYPERFLEET_LOG_*` (the comment that currently reads "Logging
configuration (can be overridden by LOG_* env vars or --log-* flags)") so
operators see the correct env-var prefix; leave the rest of the comment and the
`--log-*` flag text unchanged.

In `@internal/config/config.go`:
- Around line 72-75: BrokerConfig currently only defines Topic but the tests
expect broker.subscription_id; add a SubscriptionID string field to the
BrokerConfig struct (alongside Topic) and annotate it with
yaml:"subscription_id,omitempty" and mapstructure:"subscription_id" so config
file, env and flag loading (tests referencing broker.subscription_id,
HYPERFLEET_BROKER_SUBSCRIPTION_ID / BROKER_SUBSCRIPTION_ID, and CLI flags) can
populate it; update the struct definition in internal/config/config.go
(BrokerConfig) accordingly.

In `@test/integration/test-config-loading.sh`:
- Around line 68-69: The ROOT_DIR calculation is wrong (it only ascends one
level from SCRIPT_DIR); update how ROOT_DIR is derived so it points to the
repository root—for example, change the computation of ROOT_DIR (currently using
SCRIPT_DIR and "..") to ascend two levels (../..) or use a repo-root command
(e.g., git rev-parse --show-toplevel) so references to ROOT_DIR (used to call
$ROOT_DIR/bin/sentinel and run make build) resolve correctly; update the
assignment where ROOT_DIR is defined to use the corrected path resolution.

---

Duplicate comments:
In `@cmd/sentinel/main.go`:
- Line 379: The config-dump currently marshals cfg directly which can leak
secrets; instead create and marshal a redacted copy (e.g., call a helper like
redactConfig(cfg) or cfg.Redacted()) that copies the config and replaces
sensitive fields (authorization/header maps, API keys, tokens, secret strings,
TLS private keys, etc.) with placeholder values before calling yaml.Marshal;
update the code around the data, err := yaml.Marshal(cfg) line to use the
redacted copy so terminal/CI logs never contain raw secrets.

---

Nitpick comments:
In `@charts/values.yaml`:
- Around line 109-113: The values.yaml defines config.clients.broker.topic but
the configmap template (charts/templates/configmap.yaml) is reading
.Values.broker.topic, causing the former to be unused and confusing users;
either remove config.clients.broker.topic from values.yaml or update the
configmap template to read the topic from .Values.config.clients.broker.topic
(replace occurrences of .Values.broker.topic with
.Values.config.clients.broker.topic) so the intended field is actually applied.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ace1ed10-b248-4ffa-8bff-8c54bcc57c88

📥 Commits

Reviewing files that changed from the base of the PR and between 64dcfec and 361a1cd.

📒 Files selected for processing (24)
  • README.md
  • charts/templates/configmap.yaml
  • charts/templates/deployment.yaml
  • charts/values.yaml
  • cmd/sentinel/main.go
  • configs/dev-example.yaml
  • configs/gcp-pubsub-example.yaml
  • configs/rabbitmq-example.yaml
  • docs/configuration.md
  • docs/running-sentinel.md
  • internal/client/client.go
  • internal/client/client_test.go
  • internal/config/config.go
  • internal/config/config_loading_test.go
  • internal/config/config_test.go
  • internal/config/testdata/full-workflow.yaml
  • internal/config/testdata/message-data-blank-id.yaml
  • internal/config/testdata/minimal.yaml
  • internal/config/testdata/unknown-field.yaml
  • internal/config/testdata/valid-complete.yaml
  • internal/sentinel/sentinel.go
  • internal/sentinel/sentinel_test.go
  • test/integration/integration_test.go
  • test/integration/test-config-loading.sh

@rh-amarin rh-amarin force-pushed the HYPERFLEET-549 branch 2 times, most recently from 454c33a to f61b125 Compare March 30, 2026 07:01
@rh-amarin rh-amarin marked this pull request as ready for review March 31, 2026 16:10
@openshift-ci openshift-ci bot requested review from pnguyen44 and rafabene March 31, 2026 16:10
// Validate validates the configuration
func (c *SentinelConfig) Validate() error {
if c.Sentinel.Name == "" {
return fmt.Errorf("sentinel.name is required")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Category: JIRA

Per HYPERFLEET-549 acceptance criteria and the configuration standard, validation errors should include remediation guidance showing how to fix the issue via flag, env var, or config file.

Currently all errors are bare messages like "sentinel.name is required". The standard's example shows:

Configuration validation failed:
  - Field 'sentinel.name' failed validation: required
    Please provide via:
      • Flag: --name
      • Env:  HYPERFLEET_SENTINEL_NAME
      • File: sentinel.name

This applies to all return fmt.Errorf(...) calls in Validate() (lines 259-293): sentinel.name, resource_type, clients.hyperfleet_api.base_url, poll_interval, message_decision, and message_data.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

implemented

@rafabene
Copy link
Copy Markdown
Contributor

Category: Inconsistency — charts/templates/deployment.yaml line 75

The broker topic is now set in two places: as HYPERFLEET_BROKER_TOPIC env var in deployment.yaml, and as clients.broker.topic inside the configmap YAML (both rendered from {{ tpl .Values.broker.topic . }}). Since LoadConfig binds HYPERFLEET_BROKER_TOPIC via v.Set(), the env var always wins and the configmap value is dead config. Consider removing the topic from the configmap to avoid operator confusion, or removing the env var and relying solely on the config file.

@rh-amarin rh-amarin force-pushed the HYPERFLEET-549 branch 2 times, most recently from 79b5fcf to 7ba0b96 Compare March 31, 2026 18:41
@rh-amarin
Copy link
Copy Markdown
Contributor Author

Category: Inconsistency — charts/templates/deployment.yaml line 75

The broker topic is now set in two places: as HYPERFLEET_BROKER_TOPIC env var in deployment.yaml, and as clients.broker.topic inside the configmap YAML (both rendered from {{ tpl .Values.broker.topic . }}). Since LoadConfig binds HYPERFLEET_BROKER_TOPIC via v.Set(), the env var always wins and the configmap value is dead config. Consider removing the topic from the configmap to avoid operator confusion, or removing the env var and relying solely on the config file.

I removed the env variable from the deployment

TracingEnabled: true,
Log: LogConfig{
Level: "info",
Format: "json",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Category: Inconsistency

NewSentinelConfig() defaults Log.Format to "json" (line 128), but docs/config.md line 64 documents the default as text, and the HyperFleet logging specification also says the default is text.

Either change the code default to "text":

Log: LogConfig{
    Level:  "info",
    Format: "text",
    Output: "stdout",
},

Or update docs/config.md to say json and document why the sentinel deviates from the logging spec default.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will change the default to be JSON, also in the standard

}

// validationErr returns a validation error for the given field with remediation guidance.
func validationErr(field, reason string) error {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Category: JIRA

The HYPERFLEET-549 acceptance criteria say validation errors must include "actual values", and the configuration standard shows examples like Value 70000 exceeds maximum allowed value of 65535.

validationErr only takes field and reason — there's no way to include the actual value. For "required" fields the value is implicitly empty, but for invalid values (e.g. poll_interval: -1s) showing the actual value helps operators. Consider adding an optional actualValue parameter:

func validationErr(field, reason string, actualValue ...string) error {
    var b strings.Builder
    fmt.Fprintf(&b, "Field '%s' failed validation: %s\n", field, reason)
    if len(actualValue) > 0 && actualValue[0] != "" {
        fmt.Fprintf(&b, "  Actual value: %s\n", actualValue[0])
    }
    // ... remediation hints
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this... even though poll_interval seems the only config setting benefiting from it

dev-example.yaml Outdated
@@ -0,0 +1,115 @@
# Sentinel Configuration Example - Development
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Category: Pattern

This file duplicates configs/dev-example.yaml almost line-for-line. They already diverge: configs/dev-example.yaml sets tracing_enabled: false but this root copy doesn't include it at all.

The project convention keeps config examples under configs/, and the configuration standard references ./configs/config.yaml as the dev default. Consider removing this root-level copy to avoid config drift, or if it serves a different purpose (e.g., quick-start from the repo root), add a comment at the top explaining why it exists separately.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ouch! good catch, that file was a local copy

@rh-amarin rh-amarin force-pushed the HYPERFLEET-549 branch 6 times, most recently from f3775f2 to 68c9537 Compare April 2, 2026 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants